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This  report  was  submitted  by  The  Aerospace  Corporation,  El  Segundo, 
Calif.  90245,  under  Contract  No.  F04701-82-C-0083  with  the  Space  Division, 
P.0.  Box  92960,  Worldway  Postal  Center,  Los  Angeles,  Calif.  90009.  It  was 
reviewed  and  approved  for  The  Aerospace  Corporation  by  A.  A.  Kopania, 
Director,  Resource  Cost  Analysis  Office.  Col.  J.  D.  Fitzgerald,  SD/AC,  was 
the  Air  Force  project  officer. 

This  report  has  been  reviewed  by  the  Public  Affairs  Office  (PAS)  and  is 
releasable  to  the  National  Technical  Information  Service  (NTIS).  At  NTIS,  it 
will  be  available  to  the  general  public,  including  foreign  nationals. 
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This  technical  report  has  been  reviewed  and  is  approved  for  publication. 
Publication  of  this  report  does  not  constitute  Air  Force  approval  of  the 
report's  findings  or  conclusions.  It  is  published  only  for  the  exchange  and 
stimulation  of  ideas. 
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SUMMARY 


The  over-all  methodology  presented  for  determining  total  system  cost  risk 
is  judged  to  constitute  a  significant  advance  in  the  area  of  cost  risk  deter- 
mination.  Although  it  admittedly  favors  pragmatic  results  over  "noncoat- 
effective"  statistical  rigor,  it  is  basically  sounds  The  fundamental  value 
of  the  methodology  is  that  it  recognises'  explicitly  and  derives  analytically 
impacts  on  cost  of  the  uncertainties  prevailing  in  both  system  element  esti¬ 
mates  and  in  cost  estimating  relationships  -  uncertainties  that  were  formerly 
ignored,  but  which  contribute  significant*^  to  cost  estimating  "error*" 
Methodologies  are  presented  for  arriving  at  the  uncertainty  surrounding 
system  element  costs  that  are  derived  by  alternative  coating  approaches 
(a.g.,  parametric,  engineering,  analogy,  and  constant  multiplier). 

It  is  shown  that  the  uncertainties  in  the  costs  of  the  system  elements 
cannot  be  validly  aggregated  by  the  conventional  technique  of  adding  the 
variances  (root-sum-squaring).  The  proposed  approach  -  which  is  recognised 
as  being  conservative  in  that  the  errors  are  not  diminished  by  the  process  of 
error  aggregation  -  consists  of  adding  the  standard  deviations  associated 
with  the  costs  of  the  system  elements.  Pragmatically,  the  estimate  of  the 
total  coat  error  tends  to  be  "reasonable."  By  generating  a  cost-probability 
curve  (truncated),  the  cost  risk  involved  can  be  perceived.  However,  for 
budgetary  purposes,  a  risk-adjusted  cost  estimate  (R-ACE),  corresponding  to  a 
probability  level  commensurate  with  the  novelty  of  the  system,  needs  to  be 
selected.  The  risk-adjusted  cost  thus  selected  should  be  used  for  the 
determination  of  budgetary  requirements.  A  management  reserve  base  on  the 
difference  between  the  R-ACE  and  the  expected  cost  (0.5  probability)  should 
be  established  inasmuch  as  the  expected  cost  is  virtually  certain  to  be 
exceeded. 

By  compiling  data  on  the  results  of  the  application  of  this  methodology, 
particularly  on  the  resultant  accuracy  of  the  selected  R-ACE  values,  specific 
guidelines  or  relationships  for  selecting  probabilities  commensurate  with  the 
novelty  of  systems  can  be  established.  Complex,  first  generation  systems 
might  necessitate  a  R-ACE  at  the  0.9  probability  level,  whereas  fifth  gener¬ 
ation  systems  might  be  accurately  estimated  at  the  0.7  probability  level. 
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Identification  of  such  probability-determining  guidelines  could  further 
improve  the  accuracy  of  system  cost  estimating* 

The  presentation  of  this  methodology  is  but  the  first  step.  Its  applica¬ 
tion  together  with  documentation  of  results ,  will  permit  the  identification 
and  correction  of  any  deficiencies.  Toward  this  end,  the  methodology  devel¬ 
oped  for  propagating  error  (uncertainty)  in  the  parametric  costing  approach 
together  with  the  methodology  for  aggregating  the  cost  uncertainty  have  been 
incorporated  into  the  Air  Force  Space  Division  computerised  version  of  the 
Unmanned  Spacecraft  Cost  Model.* 

The  potential  applicability  of  this  methodology  or  its  elements  extends 
beyond  cost  risk  estimation  per  ae.  The  methodology  for  the  identification 
of  errors i  their  propagation  through  mathematical  relationships  and  ultimate 
aggregation  appears  to  be  applicable  to  technical  fields  other  than  cost 
estimating. 

By  continuing  identification,  reduction,  and  eventually  elimination  of 
sources  of  cost  estimating  error,  more  accurate  cost  estimates  will  be 
achieved.  The  methodology  developed  and  described  herein  is  a  step  in  that 
direction.  Consistently  accurate  system  cost  estimates,  however,  should 
realistically  be  viewed  as  an  illusive  goal  because  of  the  inevitable 
presence  of  inherently  random  or  uncontrollable  events  that  impar  .  costs. 
(Even  with  complete  estimating  accuracy  at  all  steps,  a  0.9  probability  level 
cost  will  be  exceeded  10X  of  the  time  because  of  random  influences.)  Never¬ 
theless,  significant,  further  improvements  can  be  achieved  through  better 
understanding  of  the  sources  of  cost  errors  and  the  decisions  that  influence 
the  procurement  of  the  system  being  costed. 


*  Jose  Gutierres  of  The  Aerospace  Corporation  was  responsible  for  modifying 
the  cost  model  (on  the  Hewlett-Packard  9845)  such  that  both  numerical 
results  and  cost-probability  plots  would  be  generated. 


PREFACE 


The  research  culminating  in  this  report  began  several  years  ago  as  a 

personal  effort.  The  first  product  -  a  briefing  titled  A  Proposed  Generic 
Approach  to  Conceptual  Phase  Risk  Analysis  -  was  presented  at  the  Space 

Systems  Cost  Analysis  Group  (SSCAG)  meeting  at  Sunnyvale,  Calif,  on 

2.3  September  1981.  It  described  general  error  propagation  ‘techniques  and 
error  aggregation  methodologies  and  problems.  The  next  product  was  a  brief* 
ing  on  a  significant  achievement  in  error  propagation  which  was  accomplished 
with  the  aid  of  Dr.  R.  H.  Huddlestons.  This  briefing,  in  the  form  of  an 
actual  costing  exampls,  displayed  the  use  of  the  algebraic  technique 
developed  for  propagating  uncertainty  in  an  independent  variable,  through  the 
uncertainty  inherent  in  a  coat  estimating  relationship  (CER)  based  on  the 
independent  variable,  so  as  to  generate  an  estimate  of  the  uncertainty  in  the 
derived  cost.  The  briefing,  titled  Example  of  the  Deterministic  Method  for 
Generating  Co at -Probability  Relationships  was  presented  at  the  Denver  SSCAG 
meeting  on  20  January  1982.  The  application  of  this  error  propagation 

methodology  was  then  documented  and  presented  on  21  May  1982  at  the  SSCAG 
meeting  in  Stevenage,  England,  in  a  paper  titled  An  Algebraic  Technique  for 
Estimating  the  Uncertainty  of  a  Parametrically  Derived  Cost  Estimate. 

Support  of  this  effort  under  Air  Force  contract  F04701-81-C-0082 
commenced  in  August  1982.  The  focus  of  the  contracted  effort  was  threefold; 
(1)  to  develop  a  methodology  for  aggregating  the  uncertainties  in  the  cost 
slemants  so  as  to  arrive  at  an  estimate  of  the  uncertainty  in  the  total  sys¬ 
tem  cost;  (2)  to  incorporate  both  the  parametric  CER  uncertainty  estimating 
methodology  and  the  uncertainty  aggregation  methodology  into  the  Air  Force 
Space  Division's  computerised  (HP  9845)  cost  model:  and  (3)  to  document  the 
methodologiee  developed  (this  report).  Upon  outlining  the  report,  a  con¬ 
spicuous  deficiency  became  apparent.  Although  methodologies  were  presented 
for  estimating  the  uncertainties  in  system  costs  derived  by  parametric, 
engineering  (bottom-up/grasa  roots),  and  factor  approaches,  a  methodology  for 
estimating  the  uncertainty  in  a  cost  derived  by  the  analogy  approach  was 
lacking.  A  personal  effort  (nonfunded  because  it  was  beyond  the  initial 
scope  of  the  contracted  effort)  was  initiated  to  develop  this  methodology. 
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A  significant  breakthrough  in  this  effort  was  achieved  on  3  January  1983* 
again  with  the  aid  of  Dr.  Huddlestons.  A  briefing  on  cost  uncertainty 
determination  methodologies  under  the  four  costing  approaches  along  with  the 
cost  aggregation  methodology  developed  was  presented  at  the  Los  Angeles  SSCAG 
meeting  on  13  January  1983  under  the  title  Presentation  of  Methodologies 
Developed  for  Estimating  Total  System  Cost  Risk  for  Four  Alternative  Costing 
Techniques. 

This  report  documents  the  methodologies  developed  for  generating 
quantitative  estimates  of  total  system  cost  risk.  It  includes  detailed 
descriptions  and  examples  of  techniques  for  (1)  estimating  the  uncertainty  in 
costs  derived  by  the  four  basic  cost  estimating  approaches:  parametric , 
engineering  (bottom-up/grass  roots),  analogy,  and  constant  multiplier, 
(2)  aggregating  the  derived  cost  uncertainties;  (3)  generating  cost-risk 
(probability)  relationships;  and  (4)  selecting  a  R-ACE  value.  Because  cost 
estimates  and  analyses  are  frequently  generated  by  individuals  with  non- 
mathematical/anginaering  backgrounds,  descriptions  of  the  methodology  as  well 
as  the  illustrative  examples  were  written  with  these  individuals  in  mind. 

During  the  past  several  years,  a  number  of  SSCAG  members  contributed  - 
through  challenging,  informal  discussions  of  aspects  of  the  problem  -  to  the 
development  of  the  methodologies  and  views  presented.  Specifically,  the 
support  of  Col.  Lowell  Maxwell  (USAF-Ret.),  who  was  chairman  of  SSCAG  whan 
these  efforts  commenced  should  be  recognised,  as  well  as  that  of  his 
successor  as  SSCAG  chairman,  Col.  Dan  Fittgerald,  under  whose  support  this 
effort  was  concluded.  In  addition,  the  members  of  the  SSCAG  Risk  Analysis 
Task  Team  -  Robert  Black,  Edward  (Ned)  Dodson,  Alvin  Owens,  Robert  Seldon  and 
Jamas  Wilder  -  should  be  acknowledged  for  their  stimulating  and  penetrating 
comments  during  the  evolution  of  the  methodology. 
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I.  INTRODUCTION 


A.  BACKGROUND 


Virtually  all  coat  estimates  -  whathar  for  private,  commercial,  or 
governmental  ayataoi  -  ara  praaentad  aa  fixad  point  aatimataa  without  qual¬ 
ification,  and  ara  eoomonly  aaaumad  to  poaaaaa  a  high  laval  of  cartainty, 
Howavar,  avan  a  ouraory  axamination  of  tha  natura  of  coata  ravaala  that  thay 
ara  intrinaically  highly  variable.  Unlaaa  tha  coat  of  a  commodity  or  ayatam 
alamant  ia  artificially  fixad,  it  will  vary  locally,  regionally,  nationally 
and,  to  compound  tha  variability,  temporally.  Raw  material  coata,  labor 
coata,  utility  coata  all  vary,  if  not  locally,  than  regionally  and  nation¬ 
ally,*  Complex  systems  ara  aaaantially  tha  raaulta  of  labor  (managerial, 
engineering,  manufacturing,  ate.),  equipment,  and  anargy  applied  to  raw 
raateriale.  If  tha  coata  of  thaaa  elamente  poaaaaa  an  inherent  variability, 
than  tha  raaulting  aggregated  coat  of  tha  ayatama  created  would  likewiee 
diaplay  a  corraaponding  variability.  Tha  foregoing  aourcea  of  coat  uncer¬ 
tainty  ara  further  compounded  by  uncertainties  inherent  in  both  tha  apacific 
dataila  of  tha  ayatam  being  coated,  and  in  the  amounts  of  labor  and  materials 
that  will  be  required. 


*  According  to  tha  1980  Dodge  Manual .  published  by  McGraw  Hill,  the  wage 
rates  for  laborara  in  Raleigh,  N.b.  was  $6.06  par  hour,  wharaaa  430  air 
miles  away  in  Cleveland,  Ohio,  it  was  $17.79  par  hour  or  190%  greater. 
Salaries  of  profaaaional  engineers  likewise  show  considerable  variability 
as  reported  in  Professional  Engineer  Income  and  Salary  -  1981.  published  by 
the  National  Society  of  Professional  Engineers.  Baaed  on  samples  of 
hundreds  of  professional  engineers,  the  difference  in  median  salary  between 
engineers  in  Columbus  and  Dstroit  (a  distance  of  leas  than  200  miles)  was 
more  than  20%.  A  20%  median  salary  differential  was  also  ahown  between 
hundreds  of  professional  engineers  in  Houston  and  San  Antonio,  approx¬ 
imately  200  miles  apart.  In  tha  summer  of  1980,  the  residents  of  Atlanta 
paid  4.23  cents  per  kilowatt-hour  of  electricity,  whereas  New  Yorkers  paid 
11.77  cents  per  kilowatt-hour,  or  177%  more.  During  tha  same  period,  the 
rate  per  kilowatt-hour  in  Indianapolis  waa  4.96  cants  while  175  miles  away 
in  Columbus  it  waa  7.77  cents  or  57%  more.  (Baaed  on  a  survey  by  the 
National  Asaociation  of  Regulatory  Utility  Commissioners) .  Material  costs 
depend  on  extraction  and  processing  labor  costs,  processing  energy  costs, 
and  transportation  costs,  all  of  which  are  variable  geographically  and 
temporally. 


Regardless  of  what  tachniqua  may  be  uaad  to  generate  ayatam  coat  esti¬ 
mates,  the  variability  inherent  in  the  cost  of  the  system  elements,  when 
combined  with  the  intrinsic  uncertainty  in  the  coafc-impacting  details  of  the 
system  together  with  such  factors  as  inflation  impacts,  should  make  any 
unqualified  point  cost  estimate  highly  suspect.  A  coat  estimate  at  beat  ia 
only  an  approximate  repreaentation  of  the  expected  system  cost  rather  than 
the  precise  price  that  it  is  too  commonly  assumed  to  be. 

B.  PURP08E/0BJECTIVE 


This  report  presents  quantitative  methodologies  for  identifying  the  cost 
variability  within  the  individual  system  elements,  a  methodology  for  aggre¬ 
gating  the  estimated  uncertainties  in  the  costs  of  the  system  elements,  and  a 
methodology  for  depicting  the  system  cost  uncertainty  by  a  range  of  system 
coats  with  their  associated  probabilities  of  occurrence.  Thus,  the  inherent 
uncertainties  prevailing  in  a  system  and  in  its  costs  would  be  reflected  by  a 
cost-probability  relationship  rather  than  as  an  ambiguous  point  cost  estimate. 

Two  basic  techniques  are  currently  used  for  generating  the  desired 

cost-probability  relationships:  the  Monte  Carlo  approach  and  the  Method  of 
12  3 

Momenta  approach.  '  '  Both  of  these  methods,  however,  are  of  such  com¬ 
plexity  that  the  use  of  a  computer  (or  at  least  a  programmable  calculator)  is 
essential.  Approximately  300  computer  iterations  are  required  by  the  Monte 
Carlo  method  in  order  to  generate  a  relatively  smooth  cost-probability 
relationship.  The  need  for  a  simple,  straightforward  technique  for  gen¬ 
erating  a  quantitative  estimate  of  the  cost  uncertainty  (risk)  has  long  been 
recognised.  The  objective  of  this  report  is  to  describe,  in  detail,  the 
direct,  quantitative  methodologies  developed  for  deriving  subsystem  cost 
uncertainties,  aggregating  these  uncertainties  into  a  total  system  cost 
uncertainty,  and  finally  deriving  a  probability-related  coat  estimate. 
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II.  CONTENTS  AND  SCOPE 


The  subsequent  methodology  for  determining  total  system  cost  risk  con¬ 
sists  of  three  distinct  sequential  steps  or  areas.  The  first  area  focuses  on 
methods  for  estimating  the  uncertainty  associated  with  the  cost  of  individual 
system  elements.  Methods  are  developed  and  illustrated  for  determining  these 
cost  uncertainties  when  the  cost  estimates  are  derived  by  any  of  the  four 
basic  costing  approaches:  parametric,  engineering  (bottom-up/grass  roots), 
analogy,  or  constant  multiplier  (factor).  The  major  sources  of  uncertainty 
that  are  ultimately  manifested  in  cost  uncertainty  are  treated  within  each 
costing  approach.  These  sources  of  cost  uncertainty  are  identified  in  Table 
1  for  each  of  the  four  costing  approaches.  In  the  parametric  costing 
approach,  the  two  major  sources  of  uncertainty  considered  are  the  uncertainty 
in  the  independent  parameter  (a.g.,  subsystem  weight*)  and  the  uncertainty  in 
the  cost  estimating  relationship  (CBR).  In  the  engineering  approach,  the 
uncertainty  in  the  man-hours  (or  quantity  of  material  required)  is  considered 
as  well  as  the  uncertainty  in  the  corresponding  wage  rate  (or  cost  per  unit 
quantity  of  material).  In  the  aralogous  or  scaling  approach,  the  uncertainty 
in  the  system  parameter  (e.g.,  weight,  power)  and  the  uncertainty  in  the 
scaling  exponent  are  considered.  In  the  constant  multiplier  or  factor 
approach,  the  only  source  of  uncertainty  is  in  the  estimate  of  the  system 
parameter  which  is  multiplied  by  a  fixed  factor  to  obtain  the  estimated 
cost. 

Factors  contributing  to  cost  uncertainty,  such  as  technological  develop¬ 
ments  required,  funding  stretch-out  or  schedule  slippage,  and  design  changes 
are  not  specifically  considered.  However,  they  are  intrinsically  incorpor¬ 
ated  in  a  normalised  manner  within  the  historic  data  points  on  which  the 
parametric  CERs  are  baaed  and  also  in  a  similar  manner,  in  the  known  system 
cost  (C2),  and  within  the  exponent  in  the  analogous  system  costing  approach. 


*  Weight  really  is  a  dependent  variable,  but  because  cost  is  often  highly 
correlated  with  it,  it  is  frequently  used  in  CERs  as  an  expedient 
surrogate  for  a  normalised  mix  of  independent  system  parameters,  which  are 
also  highly  correlated  with  it. 
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Table  1.  Major  Source*  of  Coat  Uncertainty  for  Four  Basic  Costing  Approaches 


System  Element 

Costing  Approach  Equation _  Sources  of  Uncertainty 


1.  Parametric 


2 .  Engineering 


3.  Analogy 


4.  Factor 


Cost  ■  a  *  bPc 


System  parameter  P 

Cost  model  (std.  error  of  eat.) 


Cost  «  (MH)  x  (Rate)  -  Man-hours  required 
Cost  *  (Mat)  x  (Price)  -  Rate  (skill  level) 

-  Material  required  (quantity) 

-  Price  (per  unit  quantity) 


Cost  ■  P  x  K 


System  parameter  P^ 
Scaling  exponent  Y 

System  parameter  P 


The  second  area  in  the  overall  methodology  development  preaents  a  pro¬ 
cedure  for  aggregating  the  individual  system  element  cost  uncertainties  that 
were  derived  by  one  of  the  four  approaches.  The  result  of  the  aggregation  is 
a  composite  uncertainty  that  reflects  the  uncertainty  (risk)  associated  with 
the  total  system  cost.  Application  of  the  aggregation  methodology  developed 
is  illustrated  by  an  example. 

The  third  area  presents  a  methodology  for  developing  a  cose  versus  prob¬ 
ability  relationship.  It  also  includes  some  guidelines  for  deriving  a  mean¬ 
ingful,  specific,  R-ACE  from  the  cost-probability  relationship  for  program 
funding  purposes.  The  methodologies  developed  are  then  summarised,  and 
future  efforts  for  enhancing  cost  estimating  methodologias  are  identified. 


III.  METHODOLOGIES  FOR  ESTIMATING  COST  UNCERTAINTY  WITHIN  SYSTEM  ELEMENTS 

The  methodologies  for  astimsting  the  cost  uncertainty  within  the  individ¬ 
ual  system  elements  under  the  four  approaches  shown  in  Table  1  depend  on 
estimates  of  the  mean  x  and  standard  daviation  o  of  the  cost-driving  parameters. 
The  following  section  describes  two  techniques  for  generating  values  for  the 
maan  and  standard  deviations  from  engineering  estimates  of  the  cost-driving 
parameters.  The  two  are  the  beta  distribution  and  the  triangular 
distribution  techniques. 

A.  TECHNIQUES  FOR  GENERATING  QUANTITATIVE  ESTIMATES  OF  COST-DRIVING 

VARIABLES  AND  THEIR  UNCERTAINTIES 

1.  BETA  DISTRIBUTION  TECHNIQUE 

This  technique  traces  its  origin  to  PERT  (Program  Evaluation  and  Review 
Technique)  where  it  was  widely  used  to  generate  time  eetimetes  for  events  in 
a  scheduling  network.  In  the  beta  distribution  technique  as  used  hare*  three 
estimates  of  a  system  cost-driving  parameter  are  solicited  from  the  knowl¬ 
edgeable  engineering  specialist  most  knowledgeable  about  that  system:  low, 
most  likely,  and  high.  The  low,  or  optimistic,  estimate  should  correspond 
approximately  to  a  value  that  would  be  realised  only  under  the  most  fortu¬ 
itous  circumstances  -  a  subjective  probability  somewhere  in  the  0.01  to  0.10 
range.  The  most  likely  estimate  is  just  that  -  the  mode.  The  high,  or 
pessimistic,  estimate  should  correspond  to  a  value  that  reflects  the  ultimate 
working  of  Murphy's  Law  -  a  subjective  probability  in  the  0.99  to  0.90 
range.  Thus,  if  the  three  values  are: 

a  ■  low  estimate 
m  ■  most  likely  estimate 
b  ■  high  estimate 
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then  the  mean  value  x  can  ba  aatimatad  by 

x*y(a+4m+b)  (1) 

and  the  standard  deviation  about  tha  mean  can  ba  aatimatad  by: 

ffx  "  i  <b  ”  a>  (2) 

2.  TRIANGULAR  DISTRIBUTION  TECHNIQUE 

Tha  triangular  distribution  technique  is  generally  preferred  over  the 

4 

more  connon  beta  distribution  technique .  In  the  triangular  technique,  the 
identical  estimates  -  low  (a),  most  likely  (m),  and  high  (b)  -  estimates  of 
the  cost-driving  parameter  (e.g.,  weight)  are  obtained  from  the  knowledgeable 
systam/subaystam  specialist.  The  mean  x  can  then  be  estimated  by 

*  ■  y  (a  ♦  a  ♦  b>  (3) 

and  tha  standard  deviation  o  by 

0  *  J yj  [(b  -  a)2  ♦  (m  -  a)(m  -  b)]  (4) 

Generally,  subsystem  characteristics,  performance  requirements  or  design 
constraints  enable  the  knowledgeable  subsystem  specialist  to  readily  arrive 
at  the  low  and  high  estimates.  The  most  likely  estimate  usually  presents  the 
greatest  difficulty  because  of  a  significant  degree  of  uncertainty  (indiffer- 
ance)  in  tha  broad  mid-range  area.  The  nature  of  the  computations  for  x  and 
o,  however,  tenda  to  ba  forgiving  and  reduces  the  percentage  of  error  in 
estimating  m.  Figure  1  shows  the  general  uncertainty  or  indifference  in  pro¬ 
viding  an  estimate  of  the  most  likely  value,  m. 


A  measure  of  the  sensitivity  of  x  and  o  to  the  selection  of  m  may  be 
seen  from  an  analysis  of  the  three  triangular  distributions  shown  in 
Figure  2.  It  assumes  that  curve  A  is  selected  when  the  actual  correct  values 
are  depicted  by  curves  B  or  C.  Table  2  shows  tha  sensitivity  of  tha  calcu¬ 
lated  values  of  x  and  o  with  respect  to  the  assumption  of  m. 
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Figure  1,  Schematic  of  Ooaoral  Uncertainty 

In  Eatlaeting  the  Moat  Likely  Value 


Figure  2.  Assumption  of  Most  Likely  Values 
at  23%,  30%,  and  73%  of  Range 


Table  2.  Sensitivity  of  Calculated  7  and  a 
with  Respect  to  Assumption  of  m 


Curve 

a 

m 

Am 

b 

X 

Ax 

0 

AO 

A 

30 

93 

0 

140 

93 

0 

18.4 

0 

B 

30 

72.5 

♦  31. OX 

140 

87.5 

♦  8.6X 

19.1 

3.7% 

C 

30 

117.3 

-  19. IX 

140 

102.3 

-  7.3X 

19.1 

3.7X 

The  refinement  of  intentionally  incorporating  substantial  skewness  into 
the  distribution  by  the  estimate  of  the  most  likely  value  does  not  appear  to 
be  warranted.  However,  if  there  are  recognisable  and  juatifiebla  reasons  for 
estimating  the  most  likely  value  at  other  than  approximately  the  mid-range 
point,  this  certainly  should  be  done. 

B.  PARAMETRIC  COSTING  APPROACH 

This  section  describes  and  illustrates  an  algebraic  technique  for  esti¬ 
mating  the  uncertainty  within  a  parametrically  derived  cost  estimate. 

1 .  PROBLEM 

Parametrically  derived  cost  estimates  generally  have  two  major  sources  of 
uncertainty:  the  uncertainty  inherent  in  tlie  parametric  dascriptor  of  the 
system  (e.g.,  weight),  and  the  uncertainty  associated  with  the  cost-esti¬ 
mating  regression  equation,  which  uses  the  parametric  dascriptor  to  derive 
the  cost  estimate.  The  interrelationship  between  these  two  basic  sources  of 
uncertainty  is  shown  in  Figure  3. 

The  essence  of  the  problem  is  how  to  propagate  the  uncertainty  in  the 
independent  parameter  that  is  used  to  predict  the  cost,  through  the  uncer¬ 
tainty  in  the  cost  estimating  equation,  so  as  to  obtain  an  estimate  of  the 
resultant  composite  uncertainty  in  the  derived  cost  estimate. 


STANDARD 
ERROR  OF 
ESTIMATE 


Figure  3.  Schematic  Illustration  of  fcho  Uncertainty  Propagation  Problem 


2.  ASSUMPTIONS / RKQUI RBMKNTS 

The  following  aeeumptione  are  implieit  iu  the  propoead  methodology: 

a.  An  eetimate  of  the  uncertainty  in  the  independent  variable  (e.g. , 
weight)  ean  be  generated  (usually  by  beta  or  triangular  distribution 
techniques  such  as  described  in  the  preceding  section). 

b.  A  measure  of  the  uncertainty  in  the  post-predictive  regression  equa¬ 
tion  is  obtainable  in  terms  of  the  standard  error  of  estimate.*'*'7'8 

c.  The  uncertainty  distributions  associated  with  both  the  independent 
variable  and  the  regression  aquation  are  symmetrical.  (Neither  the 
Monte  Carlo  approach  nor  the  Method  of  Moments  approach  is  restricted 
to  this  assumption.) 


d.  Tha  independent  variable  is  within  th*  ptsdictivs  rang*  (bounds)  of 
ths  rsgrsstion  aquation.  Extrapolation  beyond  tha  bounds  of  the 
ragrassion  aquation  could  introduca  significant  additional  sources  of 
uncertainty  that  ara  not  treated  hare. 

a.  Tha  coat-predictive  equation  is  of  tha  general  forms 

Y  -  a  ♦  bx° 


For  example , 

Y  ■  a  +  bxc 

Y  ■  bxe  (a,  tha  Y  intercept  is  equal  to  saro) 

Y  ■  a  +  bx  (c  is  equal  to  1) 

Y  “  bx  (a  is  aqual  to  saro,  and  c  is  equal  to  1) 

3.  METHODOLOGY  FOR  DETERMINING  THE  UNCERTAINTY  IN  A  PARAMETRICALLY  DERIVED  COST 

ESTIMATE 

Tha  application  of  tha  algabraio  relationship  requires  the  following  dates 

a.  For  the  independent  variable  (e.g.,  system  weight),  an  estimate  of 
the  mean  value  x,  and  an  estimate  of  the  standard  deviation  ox. 

b.  A  cost  predicting  regression  equation  (of  the  general  form  Y  ■  a  + 
bx°)  together  with  an  estimate  of  the  standard  error  (SB)  about  the 
regression  curve. 

Knowing  both  the  mean  and  standard  deviation  of  the  independent  variable 
(e.g.i  weight),  as  well  as  the  regression  line  equation  with  its  standard 
error  of  estimate.  Then  the  standard  deviation  of  the  dependent  variable 
(e.g.,  coat)  0y  can  be  estimated  by  the  relationships* 

0y  -  SE2  +  ^  [<x  ♦  Cx)c  -  (x  -  0X)°J  (3) 


*  R.  H.  Huddlestons,  Prediction  Error  Statistics  for  a  Nonlinear  Cost 
Risk  Model.  The  Aerospace  Corporation,  Interoffice  Correspondence 
< 1 3  January  1982).  (Not  available  for  public  distribution). 
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It  ahould  b«  noted  that  thia  relationship  ia  independent  of  a  (the  Y 
intercept)  and  ao  will  hold  true  for  regreaaion  equations  of  the  form  Y  ■ 
bxc .  In  the  linear  case  Y  '  ■  a  +  bx,  it  reduces  to  a  standard  result.  '■The 
application  of  this  methodology  is  illustrated  by  the  following  example. 

4.  APPLICATION  OF  PARAMETRIC  COST  UNCERTAINTY  DETERMINATION 


The  following  example  is  based  on  the  Telemetry,  Tracking  and  Command 
(TT&C)  subsystem  data  for  generating  first  unit  cost  and  was  extracted  from 
the  Unmanned  Spacecraft  Cost  Model.  8D-TR-81-45,  dated  June  1981  which  was 
prepared  by  Air  Force  Space  Diviaion/ACC.  The  TT&C  subsystem  design  weight 
of  81.6  lb  found  on  page  VII-3  of  the  subject  document  was  modified  by  the 
data  for  triangular  distribution  parameters  shown  in  Table  D-l,  page  D-4,  as 
follows  so  as  to  represent  typical  data  that  might  be  encountered  in  an 
actual  caset 

Basic  design  weight:  81.6  lb 

Low  estimate  (-  SOX)  40.8  lb  ■  a 

Mode  or  most  likely  estimate  (+  2.9X)  84.0  lb  ■  m 

High  estimate  (+  58X)  128.9  lb  -  b 

The  mean  value,  assuming  a  triangular  distribution,  can  be  derived  by  Eq. 

(3),  so  that  the  mean  weight  in  this  case  is 

x  ■  1/3  (40.8  +  84.0  +  128.9)  ■  84.6  lb 

Likewise,  the  standard  deviation  for  a  triangular  distribution  can  be 
estimated  by  Eq.  (4),  which  in  this  case  is 

&x  -  /  l/18f(  128.9  -  40, 8)2  +  (84  -  40.8)(84  -  128.9)]-  18.0  lb. 

The  first  unit  cost-predicting  regression  equation  for  TT&C  is  given  on 
page  IV-ll  of  the  referenced  document  to  be 

Y  ■  42.43  +  33.93x’93 


where  Y  *  First  unit  cost  in  thousands  of  FY  79  dollars  and 
x  ■  TT&C  weight  in  pounds 


By  substituting  the  mean  weight  of  84.6  lb  for  x  in  this  equation  and 
solving,  an  estimated  first  unit  cost  of  Y  ■  42.43  +  35.93(84.6)*^  ■ 
$2270.4  is  obtained.  The  standard  error  of  estimate  (SE)  for  this  equation  is 
reported  as  713.9.  By  inserting  the  corresponding  values  into  the  cost  uncer¬ 
tainty  estimating  equation  (5),  the  standard  deviation  for  the  TT&C  first  unit 
cost  is  found  to  be 


0V2  -  713. 92  ♦  21jil2£(84,6  +  18. 0)*93  -  (84.6  -  18. 0)*93]  2 
or 

oY  11  $839.2 

The  contribution  of  the  uncertainty  in  the  weight  (ox  ■  18.0  lb) 
resulted  in  an  increase  in  uncertainty  of 

*,39'?tLs  13,1,9  *  100  “  l7,6* 

over  that  attributable  to  the  standard  error  of  estimate  alone. 

C.  ENGINEERING  (BOTTOM-UP /GRASS  ROOTS)  COSTING  APPROACH 

The  engineering  approach  is  probably  the  most  widely  used  technique  for 
preparing  cost  estimates  of  systems  that  are  in  the  latter  stages  of 
development. 

1.  PROBLEM 

Engineering  cost  estimates  focus  primarily  on  labor  and  material  costs  and 
secondarily  on  energy  and  processing  costs.  Cost  estimates  of  system  elements 
are  usually  derived  by  multiplying  the  quantity  estimated  to  be  required  times 
a  unit  rata  (e.g.,  man-hours  times  dollars  per  hour;  pounds  times  dollars  per 
pound,  units  times  dollars  per  unit).  In  this  case,  the  two  major  sources  of 
uncertainty  ere  the  uncertainty  in  the  estimate  of  the  required  quantity 
(e.g.,  man-hours,  pounds),  end  the  uncertainty  in  the  estimate  of  the  cost  per 
unit  (e.g.,  dollars  per  man-hour,  dollars  par  pound).  The  essence  of  this 
problem  is  how  to  combine  the  uncertainty  in  the  quantity  required  with  the 
uncertainty  in  the  dollar  rate  per  unit  quantity  so  as  to  obtain  an  estimate 
of  the  uncertainty  in  the  product  cost. 
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2.  ASSUMPTIONS/ REQUIREMENTS 


I 


The  following  are  required  for  Che  implementation  of  this  uncertainty 
propagating  methodology: 

a.  An  estimate  of  the  quantity  required  a  along  with  a  measure  of  the 
uncertainty  in  thia  estimate  in  terms  of  the  standard  deviation 
Og*  The  estimated  standard  deviation  can  be  derived  by  either 
beta  or  triangular  distribution  techniques  as  previously  described* 

b.  An  estimate  of  the  rate  per  unit  quantity  b  together  with  an  estimate 
of  its  standard  deviation  ob«  This  too  can  be  derived  by  beta  or 
triangular  distribution  techniques  as  previously  mentioned. 

o.  The  uncertainty  distributions  associated  with  both  the  quantity  esti¬ 
mate  and  the  rate  per  unit  quantity  are  symmetrical. 

d.  The  quantity  and  rate  per  unit  quantity  are  assumed  to  be  independent. 

3.  METHODOLOGY  FOR  DETERMINING  THE  UNCERTAINTY  IN  AN  ENGINEERING 

(BOTTOM-UP/CRASS  ROOTS)  COST  ESTIMATE 

As  stated  in  the  assumptions,  in  order  to  apply  this  methodology,  it  is 
necessary  to  have  estimates  of  the  quantity  (mean)  a  and  its  standard  devia¬ 
tion  a^,  as  well  as  the  rate  per  unit  quantity  (mean)  b  and  its  standard 
deviation  o^.  These  values  are  used  to  calculate  the  coefficient  of 
variation,  (called  the  "fractional  standard  deviation"  by  some  authors)  for 
both  the  quantity  and  the  rate.  The  coefficient  of  variation  S  is  the 
standard  deviation  divided  by  the  mean.  Thus, 

o 

Coefficient  of  variation  for  quantity:  8^"  —  (6) 

o . 

Coefficient  of  variation  for  ratal  8^" 
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The  cost  is,  then,  the  product  (P$)  that  results  from  multiplying  the  quantity 
a  times  the  rate  per  unit  quantity  b 

P$  ■  a  x  b 

The  standard  deviation  Op  about  the  product  P$  is  then  found  by: 

0-  -  ab  /sfTs 1  (7) 

r  a  D 

4.  APPLICATION  OF  COST  UNCERTAINTY  DETERMINATION  IN  THE  ENGINEERING  APPROACH 


If  5000  diverse  engineering  man-hours  a  were  expected  to  be  required  to 
develop  a  specific  element  with  a  standard  deviation  of  300  man-hours, 
and  the  mean  burdened  rate  per  engineering  man-hour  b  was  estimated  to  be  $60 
per  hour  with  a  standard  deviation  of  $10  per  hour,  then  the  estimated 
cost  of  developing  the  element  is 

P$  ■  a  x  b 
P$  -  5000  x  60 
P$  -  300,000 

The  coefficients  of  variation  are 

.  °a  500  „  , „ 

Quantity:  Sa  -  —  -  -  0.10 

sb-r'i§-0<167 

and  the  standard  deviation  for  the  product  of  a  times  b  is 

0p  -  .b/  8*  ♦  8* 

-  3000  x  6o/ 0.102  +  0.1672 
ap  -  $58,400 
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Whereas,  tha  individual  coefficients  of  variation  wara  0.10  and  0.167,  tha 
coefficient  of  variation  in  the  resulting  product  is 


°P  B  58,400 
H  300,000 


0.195 


or  greater  than  that  of  either  of  tha  factors. 


D.  ANALOGOUS  (SCALING)  COSTING  APPROACH 


Tha  analogous  or  scaling  costing  approach  is  used  in  cases  where 
sufficient  historical  data  are  lacking  to  that  naaningful/valid  CERs  cannot  be 
developed  by  statistical  (regression)  techniques,  but  where  there  is  a  close 
similarity  between  tha  system  (subsystem)  being  costed  and  an  existing  system 
(subsystem)  whose  cost  is  known.  This  approach  is  widely  used  in  the  chemical 
process  industry  to  generate  preliminary  cost  estimates  for  new  chemical 
plants  or  equipment  based  on  the  known  cost  of  a  pilot  plant  or  smaller  scale 
installation.  It  is  commonly  referred  to  in  the  literature  as  the  "six-tenths 
factor"  costing  approach  even  though  scaling  factors  other  than  0.6  are  often 
used. 


1 .  PROBLEM 

In  the  analogous  costing  approach,  a  system  similar,  or  as  the  term 
implies  -  analogous  -  to  the  one  being  costed  exists.  Key  characteristics 
(e.g.,  weight,  output),  proven  by  experience  to  be  highly  correlated  with 
cost,  are  known  for  both  systems  -  the  analogous  system  and  the  system  being 
costed.  Thus,  the  variables  in  the  cost  estimating  equation  are 

CN  ■  cost  of  the  new  system  (sought) 

CQ  ■  known,  actual  cost  of  old,  analogous,  existing  system 
P„  ■  coat-correlated  characteristic  of  new  system 
Pq  ■  known,  cost-correlated  characteristic  of  old, 
analogous,  existing  system 
K  -  axperientially  based  scaling  exponent 
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and  the  coat  eatiraating  relationahip  ia 


(  Chirac tar i» tic  of  Naw  System 
Charactariatic  of  Old  Syatem 


Exponent 


(8) 


or 


The  charactariatic  PQ  and  coat  CQ  of  tha  analogoua,  axiating  eyatem 
ara  praciaaly  known  (no  uncertainty).  Tha  two  aourcaa  of  input  uncertainty  in 
thia  approach  ara  aaaociated  with  tha  charactariatic  of  tha  naw  eyetam  being 
coated  PN,  and  with  tha  uncertainty  in  tha  acaling  exponent  K. 


A  paruaal  of  axponanta  uaad  to  coat  chemical  induatry  alamanta  ravaala 
axponanta  ranging  from  approximately  0.1  to  over  3.0,  although  moat  fall 

Q 

between  0.4  and  0.9.  Scaling  axponanta  for  fighter  or  tranaport  aircraft 
aubayatama  generally  range  from  0.70  to  1.0,  with  moat  falling  batwaan  0.8  and 


2.  ASSUMPTIONS/REQUIREMENTS 

Tha  following  aaaumptiona  and  requirement!  underlie  tha  application  of  tha 
coating  by  analogy  approach! 
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»<  An  estimate  of  tha  main  valua  of  tha  cost-correlated  charactariatic 

(e.g.,  weight,  power  output)  of  the  new  system  or  subsystem  is 
required  along  with  an  estimate  of  the  uncertainty  about  the  mean 
value  in  terms  of  the  standard  deviation  0pN. 

b.  An  expected  value  of  the  scaling  exponent  K  is  required  together  with 
its  uncertainty  in  terms  of  the  standard  deviation  oR. 

c.  The  uncertainty  distributions  associated  with  the  characteristic  Pu 
and  with  tha  exponent  K  are  both  assumed  to  be  symmetrical. 

3.  METHODOLOGY  FOR  DETERMINING  THE  UNCERTAINTY  IN  A  C08T  ESTIMATE  DERIVED  BY 

THE  ANALOGY  (SCALING)  APPROACH 

This  methodology,  developed  by  Dr.  R.  H.  Huddlestone*,  differs  from  those 
previously  presented  in  that  the  uncertainty  in  the  cost  is  not  uniquely 
determined  by  a  specific  equation,  but  rather  by  a  procedure  as  follows i  The 
two  key  sources  of  uncertainty  in  the  analogy  method  are  (1)  the  estimate  of 
the  new  system's  characteristic,  and  (2)  the  estimate  of  the  scaling  expo¬ 
nent.  Estimates  of  these,  along  with  their  standard  deviations  as  measures  of 
their  uncertainty,  need  to  be  obtained,  which  can  be  done  by  the  beta  or  tri¬ 
angular  distribution  techniques  as  described  in  Sections  III  A  1  and  III  A  2. 

The  standard  deviations  are  than  applied  to  tha  mean  values  of 
and  K  so  as  to  obtain  4-  1  and  -  1  standard  deviation  values  for  each  of  the  two 
parameters  -  four  points  in  all. 


PN*  10 
PN-  10 


K  +  10 
K  -  10U 


K 


PN 

PN 


*  R.  H.  Huddlestone,  Estimated  Error  in  Costing  by  Analogy.  The  Aerospace 
Corporation,  Interoffice  Correspondence  (7  January  1983)7  (Not  available 
for  public  distribution.) 
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That*  valuta  art  than  uaad  Co  datarmina  four  eatimataa  of  eha  coat  C^,  where 


Th*  four  valuaa  of  Cg  art  obtained  by  aubatitutlng  eha  following  four  aata 
of  valuaa  Into  eha  praoading  aquation  and  aolving  for  C^t 

1*  PN  "  laPN'  K  “  l0K 

2.  PN  -  lCpN,  K  +  10K 

3.  PN  ♦  lffpN»  K  -  loR 

4.  PN  ♦  10pN,  K  +  10R 

Th*  aaan  of  tha  four  Cg  valuaa  la  than  obtained,  and  the  atandard  deviation 
of  th*  four  valuaa  of  Cg  from  tha  mean  Cg  ia  generated.  Tha  beat  estimate 
of  tha  ooat  of  th*  new  ayatem  ia  th*  mean  value,  and  eha  atandard  deviation 
ehua  calculated  ia  tha  aoughe  aatimat*  of  the  uncertainty  in  th*  ayatam  coat. 

4.  APPLICATION  OF  COST  UNCERTAINTY  DETERMINATION  IN  THE  ANALOGY  APPROACH 

A  ayatam  analogoua  to  the  one  being  coated  waa  known  to  have  weighed 
200  lb  (PQ)  and  coat  $10  million  (CQ).  Tha  new  ayatam  ia  expected  to 
weigh  800  lb  (Pg)  with  a  atandard  deviation  of  200  lb  (Opg). 
•xpactad  acaling  axponant  Kg  ia  estimated  to  ba  0.6  with  a  atandard 
deviation  <?K  of  0.2.  Thua, 

CQ  «  $10  M 
PQ  -  200  lb 

P^  (low,  -  lopN)  -  600  lb 
P^  (expected)  ■  800  lb,  crpg  ■  200 
(high,  +  lopH)  -  1000  lb 
and  K^  (low,  -  lOg)  ■  0.4 

Kg  (axpactad)  ■  0.6,  Cg  ■  0.2 
Kh  (high,  +  1(?K)  -  0.8 
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By  using  the  equation 


and  tha  four  preceding  seta  of  value*  for  PN  and  K,  four  estimates  of  CN 
are  obtained.  The  best  estimate  of  CN  can  be  shown  to  lie  within  the  bounds 
of  these  four  points,  as  illustrated  in  figure  4.  An  estimate  of  the  standard 
deviation  oQN  can  be  obtained  by  determining  the  standard  deviation  of  the 
value  of  for  these  four  points  from  the  mean  C^,  as  shown  in  Table  3. 

Table  3.  Illustrative  Data  and  Computational  Results  for  Determining 
Estimated  Cost  and  Cost  Uncertainty  by  the  Analogy  Approach 


Point 

*N 

K 

Calc. 

CN 

Dev.  (D)  From 
Mean  Cr 

D2 

1. 

Low 

600 

Low  0.4 

13.32 

8.20 

67.24 

2. 

Low 

600 

High  0.8 

24.08 

0.36 

0.13 

3. 

High 

1000 

Low  0.4 

19.04 

4.68 

21.90 

4. 

High 

1000 

High  0.8 

36.24 

12.32 

136.73 

Total  CN  ■ 

94.88 

Total  D2  - 

246.02 

Mean  CN  ■ 

23.72 

Then, 

°CK  ‘  '  ’•'* 

Thus,  the  estimated  cost  of  the  new  system  is  623. 7  million  with  a 
standard  deviation  oCN  of  $7.8  million. 

E.  CONSTANT  MULTIPLIER  (FACTOR)  COSTING  APPROACH 


In  some  instances,  the  cost  of  subsystems  can  be  estimated  by  multiplying 
the  cost  estimate  of  a  related  eyetem  element  by  a  fixed  factor(s).  The 
factor  might  reflect  a  spares  cost  factor,  inflation  factor,  learning  curve 
factor,  or  aome  other  fixed  parameter. 


0  ZOO  400  600  800  1000 

CHARACTERISTIC  (P„)  VALUES 
N 

Figure  4.  Schematic  of  Bounde  of  Syetem  Coot 


1.  PROBLEM 

In  thlo  approach,  tho  footer  io  fixed  -  by  edict  or  mathematical  princi- 
plot •  Tho  coot,  which  io  multiplied  by  tho  factor,  however,  does  poosooo  un¬ 
certainty  -  either  actual  hietorical  variability  or  developed  estimated  coot 
uncertainty.  The  problem  in  this  approach  io  to  aocortain  the  variability  in 
a  coot  developed  by  multiplying  a  coot  estimate,  that  hao  tome  level  of  uncer¬ 
tainty  by  a  fixed  eonotant. 
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2.  ASSUMPTIONS/REQUIREMENTS 


The  following  are  required  for  the  implementation  of  the  constant  multi¬ 
plier  (factor)  coating  approach: 

a.  An  eetimate  of  the  hiatorioal  or  previously  generated  (expected)  cost 
of  the  aubsystam/element,  along  with  a  measure  of  the  uncertainty 
about  the  expected  cost  as  expressed  by  the  standard  deviation* 

b.  The  uncertainty  distribution  about  the  expected  cost  is  symmetrical. 

3.  METHODOLOGY  FOR  DETERMINING  THE  UNCERTAINTY  IN  A  C08T  ESTIMATE  GENERATED 
BY  A  CONSTANT  MULTIPLIER 

The  methodology  for  propagating  the  uncertainty  in  a  variable  when  it  is 

11  12 

multiplied  by  a  constant  is  wall  established,  '  and  is  presented  here  for 
completeness.  The  methodology  consists  of  multiplying  the  standard  deviation 
of  the  original,  independent  variable  by  the  conatant  in  ordar  to  obtain  tha 
standard  deviation  in  the  resulting,  dependent  variable.  Thus,  if  the  initial 
cost  is  C,  its  standard  deviation  O,  and  the  constant  multiplier  K,  then  the 
new  cost  Cg  and  its  standard  deviation  o2  are  found  by: 


«  KCl 

(9) 

-  KOl 

(10) 

4.  APPLICATION  07  COST  UNCERTAINTY  DETERMINATION  IN  THE  C0N8TANT  MULTIPLIER 
APPROACH 

If  a  system  was  estimated  to  cost  $850  million  in  some  past-year's 
dollars  with  a  standard  deviation  of  $200  million,  and  the  inflation  factor 
to  bring  the  past  costs  to  current-year  dollars  is  1.3,  then  the  new 
(current-year)  cost  and  its  standard  deviation  are  obtained  as  follows: 
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K  *  conatant  (•.g.,  inflation  factor) 

Cj  ■  coat  in  paat-yaar  dollar* 

Cg  ■  coat  in  inflated  currant-year  dollar* 

■  uncertainty  aaaoci.nted  with  the  coat  expraaaed  in  paat- 
yaar  dollar* 

o2  ■  uncertainty  aaaociatad  with  adjuated-for-inflation  coat 


and 


C2  ■  1.3  x  $830  million 
C2  ■  $1103  million 

a2  ■  Ka1 

a 2  ■  1.3  x  $200  million 
o2  ■  $260  million 


Thua,  tha  coat  in  currant-year  dollar*  la  found  to  ba  $1103  million,  with 
atandard  deviation  of  $260  million# 


IV.  METHODOLOGY  FOR  AGGREGATING  THE  COST  ELEMENT  UNCERTAINTIES 


In  the  preceding  section,  methodologies  were  presented  for  developing 
cost  uncertainties  when  individual  system  elements  ere  costed  by  one  of  the 
four  basic  costing  approaches.  The  costs  of  the  individual  system  elements 
are  obviously  aggregated  by  adding,  but  it  is  not  clear  how  the  uncertainties 
(e.g.,  as)  associated  with  the  system  element  costs  should  be  combined  to 
reflect  the  uncertainty  in  the  aggregated  cost.  This  section  identifies  a 
methodology,  along  with  its  rationale,  for  aggregating  the  cost  uncertainties. 

A.  FALLACY  IN  APPLICATION  OF  CLASSICAL/CONVENTIONAL  STATISTICS 


Conventional  statistics  holds  that  if  variables  are  additive,  then  their 
variances  (the  square  of  the  standard  deviation  or  a2)  are  also  additive. 
Thus,  the  veriance  about  the  sum  of  a  series  of  variables,  each  with  its  own 
variance,  is  found  by  adding  the  variances  as  follows: 


Standard  Deviation 


Variant 


The  variance  about  the  sum  Og  then,  is 


2  2  2  2 

a.  a.  ♦  a‘  +  o‘ 

m  D  C  a 


Tha  atandard  daviation  about  tha  aura  may  than  ba  darlvad  by  talcing 
tha  aquara  root  of  both  aidaa  of  tha  aquation;  thua, 

/  2  2  2  2 

o-  -  /  O.  +  +  0^  +  0* 

5  a  o  c  d 

Thia  la  tha  oonvantional  root-aua-aquara  (R8S)  ralationahip.*  What 
happena,  howavar,  whan  thia  ralationahip  ia  appliad  to  ayatam  coat  data? 

Suppoaa  a  ayatam  ia  eompoaad  of  thraa  indapandant  elements,  aaoh  with  a 
coat  of  30  and  a  atandard  daviation  about  tha  coat  equal  to  10%  of  tha  coat, 
or  3.  Tha  total  ayatam  coat  and  ita  atandard  daviation  would  ba  darivad 
thualyi 


Coat 

Standard  Daviation 

1 

SO 

51 

2 

30 

*2 

3 

30 

3, 

Total  coat  ■  130 

3 

Tha  atandard  daviation  about  tha  total  coat  computed  by  applying  tha  R8S 
approaoh  would  ba 

0,  -  /s}  ♦  sjj  ♦  J*  -  8.66 

Tha  atandard  daviation,  which  initially  amounted  to  10X  of  tha  coat  of 
each  element  la,  by  tha  application  of  tha  R88  technique,  reduced  to  3.8%  of 
tha  aum. 


*  A  refinement  of  tha  RS8  taehniqua  focuaaa  on  tha  independence  or  non- 
independence  of  the  variablaa  being  aggregated.  If  they  are  indeed  all 
indapandant,  than  tha  conventional,  atraightforward  RS8  aquation  appliaa. 
Howavar,  if  they  era  nonindapandant  (corralatad) ,  than  an  additional  com¬ 
plexity  ariaaa  -  covariance  tarm(a)  need  to  ba  added  to  tha  auma  of  tha 
variancea.** 

**  R.  H.  Huddlaatona,  Bat (mated  Error  in  Total  Syatem  Coat.  Tha  Aaroapaca 
Corporation,  Interoffice  Correapondenca  1 30  November  1982)  diacuaeea  thia 
iaaua.  (Not  available  for  public  diatribution. ) 
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Now,  if  eh«  system  were  twice  at  larga  and  oonaiatad  of  six  rathar  than 
thraa  lika  alamanta,  aach  of  which  again  had  a  standard  daviation  aqual  to  10X 
of  tha  mean,  thus, 


Element 

Cost 

Standard  Deviation 

1 

30 

*1 

2 

30 

52 

3 

30 

53 

4 

30 

54 

5 

30 

55 

6 

30 

5. 

Total  Coat  ■  300 

6 

than,  tha  standard  daviation  about  tha  total  oost  using  tha  R88  approach 
would  ba 


.  .2  ,  -2.-2.  -2 
+  32  v  33  +  34  +  55 


or  12.23.  This  is  but  4. IX  of  tha  sub. 


If  tha  system  wara  truly  gigantic  and  complex,  such  that  it  consisted  of  24 
statistically  independent  alaaants  under  tha  abova  assumptions,  than  tha 
total  cost  would  ba  1200,  and  tha  standard  daviation  using  tha  root-aum- 
square  approach  would  ba  24.49,  or  now  only  2.0X  of  tha  total  cost!  These 
examples,  which  show  tha  affect  of  applying  conventional  RSS  techniques,  are 
summarised  in  Table  4  below. 


Table  4.  Potential  Impact  of  Using  RGS  Technique 
for  Cost  Uncertainty  Aggregation 


No;  of  Elements  Total  Cost 


Uncertainty  a 


Uncertainty  0 
as  X  of  Total  Coat 


Thaae  data  show  that  as  th«  syatem  becomes  larger,  (a.g.,  more  expentive  and 
complex)  the  percent  uncertainty  tend#  to  decrease!  Thie  la  contrary  to  all 
coating  experience*  Something  ia  wrong.  An  examination  of  what  ia  actually 
occurring  will  ravaal  the  source  of  this  fallacy. 

B.  SOURCE  OF  INAPPLICABILITY  OF  CONVENTIONAL  STATISTICS 

1.  NATURE  07  COMPETITIVE  PROCUREMENT  PROCESS 

When  propoaela  for  the  development  and/or  production  of  a  ayetem  are 
received  from  aeveral  qualified  contractors,  coat  ia  invariably  a  criterion 
for  selecting  the  winner.  Not  neceaearily  the  lowest  coat  -  but  rather  a 
reasonable,  credible  coat.  This  obviously  puts  pressure  on  the  competitive 
bidders  to  develop  low  but  realistic  bide.  However,  in  most  aystam  procure¬ 
ment  a,  there  is  a  significant  element  of  uncertainty.  This  ia  reflected  by 
the  inclusion  of  prototype  development  and  teats  into  the  program.  The  testa 
are  in  essence  demons tret  ions  to  ascertain  whether  the  manifold  uncertainties 
pervading  the  system  have  been  euccassfully  resolved.  Testa  are  expensive. 
How  many  teats  will  be  required  reflects  Just  one  aspect  of  contractor 
optimism.  Some  level  of  optimism  (confidence  in  the  engineering  staff's 
technical  competence)  must  be  displayed.  A  bid  based  on  pessimistic  outcomes 
would  surely  lose  to  a  more  success-oriented  competitor.  On  the  other  hand, 
the  compulsion  to  submit  e  low  bid  based  on  substantial  optimism  would  be 
prudently  balanced  by  experientially  derived  knowledge  of  the  existence  of  a 
potentially  malevolent  reality  (Murphy's  Law!).  Consequently,  modest 
reserves  for  a  limited  number  of  adverse  events  are  usually  incorporated  into 
the  bid.  Thus,  the  nature  of  the  competitive  procurement  process  results  in 
bids  that  (1)  tend  to  be  somewhat  low,  based  on  both  an  optimistic  view  of 
the  contractor's  technical  competence  and  prevailing  competitive  forces,  and 
(2)  belong  to  a  two-tailed  distribution  Inssmueh  as  the  cost  could  be  lower 
if  the  program  were  so  successful  that  the  reserves  were  not  needed;  or 
higher,  if  the  amount  of  reserves  were  inadequate.  Invariably,  the  uncer¬ 
tainties  surrounding  the  costs  of  all  of  the  individual  elements  that  are 
aggregated  to  produce  the  final  bid  are  two-tailed,  even  though  some  tails 
may  be  highly  skewed.  Thus,  in  aggregating  the  costs  and  their  uncertain¬ 
ties,  a  root-sum-square  approach  would  appear  valid.  The  high-tailed  cost 
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uncertainties  would  be  offset  by  tha  low-tailed  coat  uncertainties.  Tha 
population  would  tand  towarda  normality,  and  tha  Central  Limit  Theorem  would 
hold. 

2.  MANAGEMENT  PRINCIPLES/DECISION  DRIVERS 

Once  a  contract  ia  awarded,  what  waa  up  to  that  point  a  "normal"  diafiri- 
bution  become*  a  truncated  (one-sided)  distribution,  and  tha  root-aum-aquare 
approach  to  aggregating  tha  uncartaintiae  ia  no  longer  valid*  That  portion 
of  tha  coat  diatribution  which  liaa  balow  tha  mean  or  expected  coat  dia- 
appaara,  ao  there  are  no  offaatting  valuaa  for  tha  high  portion  of  tha 
diatribution.  Tha  raaaon  for  thia  ia  baaed  on  program  management  motivatora 
or  daciaion  driven.  Tha  program  manager  ia  acutely  aware  that  tha  syatama 
being  procured  have  a  higher  intrinaie  value  than  what  ia  reflected  in  their 
coat  (or  elaa  thair  acquisition  would  ba  uneconomic  and  imprudent).  Both  hia 
and  hia  company 'a  reputation  dapend  on  the  reliable  performance  of  the 
developed  system  over  ita  expected  life  -  but  in  tha  eaae  of  apace  ayatama  - 
with  atringant  weight  and/or  voluma  eonatrainta.  The  anticipated  reliability 
ia  aubatantiated  by  failure  mode  analyaea  which  pinpoint  weak  linka. 
Enginaara  reaponaible  for  alamanta  of  tha  apace  ayatam  invariably  identify 
changaa  that  would  enhance  reliability  and  improve  tha  ayatam,  but  uaually  at 
an  incraaaa  in  coat,  weight,  and/or  voluma.  Thua,  if  tha  coat  ia  under- 
running,  a  prudent  manager  would  opt  to  uae  the  available  funda  to  eliminate 
the  evar-praaent  weakeat  link  and  ao  enhance  the  ayatam' a  reliability,  life, 
or  other  key  attributes.*  It  ia  virtually  inconceivable  that  a  program 
manager  would  decide  to  coma  in  balow  the  contract  coat  while  weak  links 
axiat  that  could  jeopardies  tha  total  success  of  the  system.  (How  much 
tasting  and  reliability  ^s  enough?) 


*  Major  General  Jasper  Welch,  USAF,  Assistant  Deputy  Chief  of  Staff  for 
Reaearch,  Development  and  Acquisition,  writing  in  tha  December  1982  issue 
of  Electronic  Business  (pages  55,  56)  admonished  contractors  for  merely 
meeting  rather  than  substantially  exceeding  negotiated  MTBF  specifications. 


Furthermore,  the  level  of  contractor  optimism  evoked  by  a  competitive 
procurement  process  operates  to  reduce  the  probability  of  such  a  highly 
successful  program  that  a  cost  underrun  would  result.  (Some  sole-source 
procurements  without  cost-depressing  competitive  pressures  have  resulted  in 
underruns,  but  this  is  deemed  to  be  due  to  the  submittal  of  a  more  realisti¬ 
cally  high  bid  than  would  normally  be  the  case  in  a  'competitive  procure¬ 
ment.)  The  pressures  on  the  program  manager  personally  and  on  the  contractor 
in  general  to  deliver  the  "best"  possible  system,  coupled  with  the  ubiquitous 
weak  links,  preclude  cost  underruns.  Conssquently,  the  system  costs  incurred 
are  not  governed  by,  nor  display,  randomness  about  the  contract  awarded 
amount.  The  RSS  technique  in  which  variances  are  summed,  assumes  a  random¬ 
ising  nature  at  work,  or  an  "averaging"  of  the  high  side  values  with  those 
below  the  expected  (i.e.,  mean)  value.  This  does  not  occur  here.  There  are 
no  "low"  values.  If  the  costs  of  the  elements  were  normally  distributed  so 
that  some  were  higher  than  expected  while  others  were  lower  (thus  producing  a 
bell  shaped  histogram  or  curve),  then  the  conventional  RSS  approach  might 
apply,  but  this  is  not  the  case.  The  awarded  amount  -  usually  the  bid  price 
(slightly  modified  during  contract  negotiations)  -  which  was  two-tailed  in 
its  development,  becomes  a  floor  once  the  award  is  made.  The  two-tailed, 
chance  element  that  underlies  the  RSS  technique  no  longer  prevails. 
Root-sum-squaring  should  not  be  used.  It  generates  wrong  estimates  of  the 
standard  deviation  because  "averaging"  does  not  occur. 

The  foregoing  assessment  of  the  program  manager  applies  squally  down  the 
organisational  line  to  the  subsystem  project  engineers  and  subcontractors. 
Once  a  cost  goal  has  been  set  or  subcontract  award  made  (based  again  on  a 
measure  of  optimism),  that  then  becomes  the  cost  floor.  The  best  subsystem 
possible  will  be  developed  for  the  designated  cost.  However,  any  adversity 
encountered,  beyond  that  anticipated  in  preparation  of  the  proposed  bid,  will 
likely  result  in  cost  overruns.  Thus,  the  bid/award  cost  will  not  be  under- 
run.  It  can  only  be  met  or  overrun.  This  negates  the  applicability  of  the 
root-sum-square  technique  for  aggregating  system  cost  uncertainties. 
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C.  RECOMMENDED  APPROACH 


The  independent  parameters  (e.g.,  subsystem  weight,  CER  standard  error  of 
estimate)  on  which  cost  estimates  are  based  possess  an  inherent  two-tailed 
probability  distribution  (±  a).  The  beta  and  triangular  distribution  tech¬ 
niques  described  provide  methods  for  estimating  a  two-tailed  uncertainty 
about  a  mean  or  expected  valuer  The  methodologies  described  for  propagating 
the  uncertainties  in  the  independent  variables  within  the  four  costing 
approaches  result  in  the  development  of  two-tailed  uncertainty  distributions 
about  each  element  or  subsystem  cost  derived.  Now,  the  purpose  of  aggre¬ 
gating  the  element /sub system  costs  and  their  uncertainties  is  to  arrive  at  a 
total,  composite  cost  along  with  its  uncertainty  that  will  reflect  actuality 
-  what  really  may  occur.  Conventionally,  aggregation  of  two-tailed  uncer¬ 
tainties  is  validly  done  by  means  of  the  RSS  technique  -  R9S  alone  if  the 
variables  whose  uncertainties  are  being  aggregated  are  independent,  RS8  plus 
a  covariance  factor  if  the  variables  are  nonindependent.  But,  as  described 
in  the  preceding  section,  forces  prevail  that  eliminate  the  lower  tail,  thus 
negating  the  applicability  of  a  RSS  approach.  The  one-sidedness  of  the  cost 
incurrence  is  the  basis  for  Che  recommended  approach.  Specifically,  in 
aggregating  the  uncertainties  associated  with  one-sided  cost  elements,  the 
uncertainties  as  reflected  in  the  standard  deviation  associated  with  each  of 
the  aggregated  coat  elements  should  be  added  to  arrive  et  a  standard 
deviation  for  the  aggregated  cost,* 

D.  APPLICATION  0?  AGGREGATION  METHODOLOGY 


The  following  example  is  based  on  an  actual  space  system  concept.  High, 
low,  and  most  likely  weight  estimates  were  obtained  for  each  subsystem 
element  from  the  respective  subsystem  specialists.  The  triangular  distribu¬ 
tion  methodology  (discussed  and  demonstrated  in  Section  III  A  2)  was  used  to 
generate  the  expected  value  (mean)  and  estimated  uncertainty  (i.e,,  standard 
deviation,  a)  for  each  of  the  subsystem  elements.  Parametric  cost  models 


*  This  conclusion  is  supported  by  analyses  performed  by  Dr,  R,  H,  Huddlestons 
and  presented  in  Estimated  Error  in  Total  System  Cost,  The  Aerospace 
Corporation,  Interoffice  Correspondence  (30  November  19(32).  (Not  available 
for  public  distribution,) 
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wore  u««d  to  determine  tha  subsystem  a lament  costs.  The  standard  deviations 
of  the  weight  estimates  were  combined  with  the  standard  error  of  estimate  of 
the  CERs  by  means  of  the  parametric  error  propagation  equation  (5),  thereby 
providing  an  estimate  of  the  standard  deviation  (uncertainty)  in  the  system 
element  cost.  Tables  5  and  6  show  tha  subsystem  costs  by  cost  element  along 
with  the  standard  deviation  for  the  non-recurring  costs  and  recurring  costs,  ' 
respectively.  Tha  coefficients  of  variation  or  fractional  standard  devia¬ 
tions  are  also  shown  as  a  percentage.  The  standard  deviations  are  aggregated 
by  summation  into  a  total  system  standard  deviation.  The  standard  deviations 
that  would  be  obtained  if  RSS  techniques  were  used  are  shown  for  comparison. 

Tha  total  system  acquisition  cost,  not  including  launch  or  operations  and 
support  costs,  is  shown  in  Table  7,  along  with  the  aggregated  cost 
uncertainty  as  reflected  by  the  standard  deviation. 
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Table  5.  Aggregation  of  System  Non-Recurring  Coats 
and  Their  Uncertainties 


Non-Recurring  Cost 

in  PY  82  Dollars 

Parametric 

Estimated 

Coefficient 

Cost  Est. 

Std.  Dev. 

of  Variation 

System  Element 

m|> 

a  m6 

_ s_ 

Mission  Equip.  (M. E.) 


Element  A 

30.0 

3.0 

6.0 

Element  B 

223.4 

33.2 

14.7 

Element  C 

36.3 

-LlL 

7.4 

M.E.  Subtotal 

311.7 

38.9 

(33.4  RSS) 

12.5 

(10.7%  RSS) 

Spacecraft  (S/C) 

Structure  &  Mach. 

12.2 

2.1 

17.2 

Thermal  Control 

3.6 

2.1 

38.3 

Electric  Power 

14.9 

2.9 

19.5 

Trk.|  Tel.  &  Comm'd 

53.1 

2.6 

4.9 

Stab.  &  Control 

19.0 

7.8 

41.1 

Aux.  Propulsion 

3.0 

-All 

30.0 

S/C  Subtotal 

103.8 

18.4 

(9.3  RSS) 

17.4 

(8.8%  RSS) 

Integration 

13.6 

1.9 

12.2 

Qual.  &  Space  Proto. 

472.1 

33.4 

11.3 

Ground  Support  Equip., 

9.6 

1.0 

10.4 

Fee 

64.0 

8.0 

12.3 

Total  Non-Recurring  978.8 


121.6 

(64.2  RSS) 


12.4 

(6.6%  RSS) 


Table  6.  Aggregation  of  System  Recurring  Coata 
and  Their  Uncertainties 


Recurring  Coats  (buy  of  eix)  in  PY  1982  Pollan 


Parametric 

Estimated 

Coefficient 

Goat  Eat. 

Std.  Dev. 

of  Variation 

System  Element 

.  —Ml— 

O  Ml 

X 

Mission  Equip.  (M.8.) 


Element  A 

36.3 

3.8 

10.5 

Element  B 

269.4 

28.4 

10.3 

Element  C 

63.6 

6.4 

10.1 

M.B.  Subtotal 

Spacecraft  (S/C) 

364.3 

38.6 

(29.4  RSS) 

10.3 

(8.0%  RSS) 

Structure  &  Mach. 

19.2 

3.6 

18.8 

Thermal  Control 

4.8 

3.4 

70.8 

Electric  Power 

66.6 

8.3 

12.5 

Trk. ,  Tel.  &  Coram'd 

93.4 

6.3 

6.6 

Stab.  &  Control 

34.8 

7.2 

20.7 

Aux.  Propulsion 

6.6 

0.3 

ill 

S/C  Subtotal 

227.4 

29.1 

(13.6  RSS) 

12e8 

(6. OX  RSS) 

Integration 

18.6 

1.9 

10.2 

Launch  Site  Support 

8.3 

1.1 

12.9 

Fee 

43.7 

4.9 

11.2 

Total  Recurring 

667.3 

75.6 

(32.8  RSS) 

11.3 

(4.9X  RSS) 
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Table  7.  System  Acquisition  Cost  and  Cost  Uncertainty  Co) 

(In  FY  82  Dollars) 


Procurement 


Estimated 

Coat 


Estimated 
Std.  Dev. 


Coeffic lent 
of  Variation 


V.  COST-PROBABILITY  (COST  RISK)  RELATIONSHIP 


Tha  objective  of  system  coat  risk  analysis  ia  to  portray  tha  financial 
raaourcaa  raquirad  to  acquire  a  givan  aye tern  in  a  conciae  yet  raaliatic 
mannar.  A  point  cost  estimate,  although  concise,  doaa  not  raaliatieally 
raprasant  tha  aotual  uneartaintias  that  prevail.  A  coat-probability  curve 
for  tha  givan  aystam  can  depict  tha  meaaure  of  coat  risk  associated  with  the 
acquisition  of  tha  system. 

A.  METHODOLOGY  FOR  GENERATING  A  COST-PROBABILITY  CURVE 


Tha  expected  (mean)  total  system  cost  and  its  standard  deviation  -  whose 
generation  was  described  and  illustrated  in  tha  previous  section  -  are 
sufficient  for  the  development  of  a  cost-probability  curve.  Tha  expected 
cost  corresponds  to  a  0.5  probability.  If  a  normal  probability  distribution 
ia  assumed,  reference  to  cumulative  probability  tables  shows  that  tha  expect¬ 
ed  cost,  +  1  standard  deviation,  corresponds  to  a  0.841  cumulative  probabil¬ 
ity.  This  means  that  the  probability  is  0.841  or  84.  IX  that  the  cost  will  be 
less  than  tha  amount  corresponding  to  the  expected  cost,  «•  1  standard  devia¬ 
tion.  Tha  expected  cost  plus  twice  the  standard  deviation  corresponds  to  a 
cumulative  probability  of  0.977  or  97.7%.  Similarly,  the  expected  cost,  -  1 
standard  deviation,  corresponds  to  a  0.159  probability  and  the  expected  cost, 
-  2  standard  deviations  corresponds  to  a  0.023  probability.  The  cost 
probability  curve  can  be  generated  by  plotting  the  above  points  for  cost 
versus  probability  on  standard  probability  paper  and  connecting  them  with  a 
straight  line. 

The  three  prerequisites  for  generating  a  cost-probability  curve  ara 

1.  The  availability  of  the  expected  (mean)  cost  with  its  standard 
deviation. 

2.  The  assumption  of  normality  (for  at  laast  tha  upper  tail  of  the 
distribution,  as  discussed  in  Section  C  below). 


3.  The  following  table  of  cumulative  probabilities  versus  standard 
deviations  (o): 


Cumulative 

Probability 

it 

Expected  Cost 

-  20 

0.0228 

2.28 

Bxpeeted  Cost 

-  lo 

0.1387 

13.87 

Bxpeoted  Cost 

(mean) 

o.sooo 

30.00 

Expected  Cost 

♦  lo 

0.8413 

84.13 

Expected  Cost 

♦  2o 

0.9772 

97.72 

B.  EXAMPLE  OF  COST-PROBABILITY  RBLATI0N8HIP  DETERMINATION 

A  coat-probability  relationship  will  be  developed  for  the  total  system 
coat  11646 .3  million,  and  standard  daviation  $197 >2  million  developed  in 
Saotion  XV  D  and  shown  in  Tabla  7.  The  ooat  probability  curve  can  be  devel¬ 
oped  by  plotting  the  following  cost  versus  percent  probability  values  on 
probability  paper,  as  illustrated  in  figure  Si 


Cost 

X  Probability 

Bxpeotad  Cost 

1646.3 

30.0 

Expected  Cost  + 

lOJ 

1646.3 

♦  197.2  - 

1B43.3 

84.1 

Expected  Cost  - 

lot 

1646.3 

-  197.2  - 

1449.1 

13.9 

Expected  Cost  ♦ 

20t 

1646.3 

♦  2(197.2)  - 

2040.7 

97.7 

Expected  Coat  - 

2ot 

1646.3 

-  2(197.2)  - 

1231.9 

2.3 

The  steeper  the  slope  of  the  curve  (i.e.,  relatively  narrow  cost  range  about 
the  expected  cost),  the  less  the  cost  risk;  the  shallower  the  slope  of  the 
curve  C i • e • ,  relatively  wide/broad  cost  range  about  the  expected  cost),  the 
greater  the  cost  risk* 
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c.  TRUNCATION  RATIONALE 

The  two  basic  reasons  why  system  costs  are  estimated  are  (1)  to  aid  in 
determining  whether  the  anticipated  benefits  warrant  the  expenditures  neces¬ 
sary  to  acquire  the  system,  and  (2)  to  determine  the  funding  that  will  be 
required  for  its  acquisition.  A  key  characteristic  of  most  advanced  systems 
is  their  relatively  large  cost  -  developmental,  production,  end  operations. 
The  funding  of  a  system  indicates  chat  the  anticipated  benefits  are  judged  to 
be  substantially  largsr  than  its  costs,  or  else  its  development  would  be 
irrational.  The  premature  loss  of  Che  benefits  of  a  key  system  oould  have 
calamitous  consequences.  Thus,  program  managers  responsible  for  the  acquisi¬ 
tion  of  systems  have  to  balance  the  dire  consequences  of  diminished  perfor¬ 
mance  or  premature  failure  against  the  expenditures  incurred  in  system 
acquisition  and,  as  discussed  in  Sections  IV  B  1  and  IV  B  2,  would  likely  opt 
for  enhancing  the  weakest  link(s)  rather  than  underrunning.  Consequently, 
the  expected  cost  (0.5  probability)  constitutes  a  floor  on  the  system  cost; 
therefore,  it  is  recommended  that  the  cost-probability  curve  be  viewed  as 
credible  only  upward  from  the  expected  cost  (0.5  probability  point). 


Although  the  cost-probability  curve  is  a  more  valid  portrayal  of  the 
prevailing  reality  than  a  point  cost  estimate,  the  budgetary  process  is 
incapable  of  treating  a  probabilistic  spectrum  of  costs.  Therefore,  the 
probabilistic  costs  depicted  on  the  cost  curve  need  to  be  translated  into  a 
meaningful  fixed  value  for  budgetary  requirements.  A  basic  guideline  for 
accomplishing  this  is  to  use  the  cost  associated  with  a  probability  that 
reflects  the  level  of  novelty  inherent  in  the  system  -  the  risk-adjusted  cost 
estimate  (R-ACE).  For  example,  the  cost  corresponding  to  a  0.6  to  0.7  prob¬ 
ability  level  could  be  used  for  systems  with  a  substantial  legacy  from  prior 
systems  versus  a  cost  corresponding  to  a  0.8  to  0.9  probability  level  for 
systems  Incorporating  a  substantial  technological  advancement.  Major  archi¬ 
tectural  and  engineering  firms  tend  to  fevor  a  0.85  probability  level  for 
cost  risk  estimates  of  new  major  construction  projects. 


The  source  of  the  eoat  increase  being  compenaated  for  by  Che  R-ACE  ia 
usually  aoma  eapecc  of  actual  or  potential  technological  deficiency(s).  The 
praaence  of  a  technological  deficiency  ia  often  referred  to  aa  "technical 
risk."  Major  categories  of  technical  risks  are  thoae  chat  are  associated 
with  both  the  RDT&E  phase  of  a  system  aa  well  as  with  its  production  phase. 
A  prerequisite  to  the  initiation  of  a  successful  RDT&E  phase  (or  its  cost 
estimation)  is  that  the  physical  laws  and  principles  on  which  the  functioning 
of  the  system  will  rely,  must  be  in  hand.  It  is  foolhardy  to  even  attempt  to 
estimate  the  cost  of  achieving  a  technical  breakthrough.  Any  cost  estimate 
of  a  system  for  which  the  basic  technology  is  not  in  hand  is  meaningless. 
The  history  of  the  nuclear  powered  aircraft  program  is  brought  to  mind. 

The  technical  risks  in  the  RDT&S  phase  are  usually  assoc iated  with  the 
efficient  and  reliable  implementation  of  known  physical  laws  and  principlea 
through  engineering  design.  Even  then,  substantial  risks  are  occasionally 
encountered  as  in  the  case  of  the  success-eluding  program  for  developing  an 
active  refrigerator  for  cooling  spacecraft  payload  sensors  (cryo-cooler).  A 
nominal  measure  of  the  impact  of  the  technology  implementation  risk  is 
recognised  as  being  incorporated  within  the  historically-derived  subsystem 
RDT&E  CERs  and  their  standard  errors  of  estimate  that  constitute  the  cost 
estimating  data  base. 

The  second  area  of  potential  technological  deficiencies  that  create  tech¬ 
nical  risk  is  associated  with  production,  specifically  that  stemming  from  a 
lack  of  manufacturing  or  testing  know-how  or  both.  For  example,  it  is  one 
thing  to  develop  and  produce  a  demonstration-of-principle  mosaic  focal  plane 
containing  16  or  even  200  detectors  under  laboratory  conditions;  it  is  quite 
another  to  mass  produce  an  operational  version  with  ssy  10  million  detectors. 
(The  "development"  of  the  production  facilities  may  present*  an  even  greater 
technical  challenge  than  the  actual  development  of  the  detector.) 

The  resolution  of  technical  risk  (i.e.,  the  elimination  of  technological 
deficiencies)  is  accomplished  through  the  acquisition  of  new  knowledge,  which 
may  be  viewed  at  a  function  of  time  (schedule)  and  funding  (cost).  Technical 
risk  per  se  has  no  unique  intrinsic  substance;  it  manifests  itself  through 
some  combination  of  cost  and  schedule  risk  -  cost,  through  the  focusing  of 


additional  skilled  scientific  and  anginaaring  talant  on  tha  surmounting  or 

alimination  of  tha  impading  daiicianciaa ;  achadule  through  tha  application 

tha  aciantific  and  anginaaring  talant  over  a  longar  pariod  of  tima  or  mora 

intensely,  which  agrin  ia  basically  a  cost  impact.  (Eiiiher  acceleration  or 

stretch-out  of  a  program  from  a  normalised,  baselined  schedule  usually 

results  in  increased  costs  baeausa  of  increased  labor  requirements.)  Coat 

impacts,  although  they  capture  tha  major  essence  of  sohadula  changes,  omit 

the  initial  operational  capability  (IOC)  consideration  or  program  criterion. 

Nevertheless,  cost  impacts  capture  most  of  the  essence  of  schedule  risk. 

Thus,  coat  impaets  constitute  an  acceptable  surrogate  for  technical  risk. 

This  is  borne  out  in  the  testimony  given  by  Deputy  Secretary  of  Defense, 

7.  C.  Carlucci,  on  9  February  1982  before  the  House  Committee  on  Armed 
13 

Services  in  which  ha  reported  the  increases  in  the  7Y  83  DOD  budget  for 
the  technical  risks  associated  with  the  RDT4E  phase  of  12  major  programs. 
The  average  percent  funding  increase  for  "technical  risk"  for  the  12  programs 
was  21. IX.  (The  median  was  between  17.7  end  21. IX).  Thus,  in  the  absence  of 
other  data  on  which  to  eatablish  a  R-ACE,  a  cost  incraasa  on  the  order  of  20X 
may  be  the  beat  estimate  possible  under  the  circumstances.  However,  experi¬ 
ence  gained  from  the  application  of  the  R-ACE  technique  would  soon  provide 
guidelines  for  selecting  probability  levels  (and  hence  costs)  commensurate 
with  the  inherent  uncertainties,  nature,  and  characteristics  of  the  system 
being  oostad. 

The  difference  between  the  cost  corrssponding  to  the  selected  risk  prob¬ 
ability  level  and  the  expected  coat  should  go  into  a  program  reserve, 
possibly  such  as  that  recommended  in  the  Total  Risk  Assessing  Cost  Estimate 
(TRACE)  concept.*4  Whereas,  tha  budgetary  funding  request  would  correspond 
to  the  cost  associated  with  the  estimated  risk  probability  level,  the 
procurement  contract  and  any  ineentive/award  fees  should  be  based  on  the 
proposed,  "expected"  coat.  Were  the  contract  award  to  be  made  at  say  the 
0.83  probability  cost  level,  the  manifold  pressures  prevailing  on  the  program 
management  to  produce  a  reliable,  high-quality  systam  would  assure  that  the 
incremental  (reserve)  funds  would  be  committed  to  eliminating  weak  links  and 
enhancing  the  system,  thereby  raising  the  probability  of  insufficient  funds 
being  available  to  overcome  any  unexpected  adversities  late  in  the  program. 
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The  likely  net  reault  would  be  en  overrun  of  the  R*ACE  amount.  To  preclude 
this,  the  reserve  should  be  in  the  tight  control  of  the  system  program  office 
or,  in  the  case  of  major  system  contracts,  in  the  cont.-cl  of  a  top  level  DOD 
service  board,  but  not  in  the  control  of  the  contractor,  who  should  endeavor 
to  meet  his  initial  cost  estimate. 


VI.  ADDITIONAL  CONSIDERATIONS  IN  APPLICATION  OP  METHODOLOGY 


A.  INFLATION  UNCERTAINTY 


Although  the  cost  risk  methodology  presented  has  deeeribed  end  demon* 
atreted  (1)  techniques  for  estimating  the  cost  uncertainty  associated  with 
the  various  coat  elements,  (2)  e  recommended  methodology  for  the  aggregation 
of  the  ooet  uncertainties,  (3)  a  methodology  for  the  generation  of 
cost-probability  curves,  end  (4)  some  initial  guidelines  for  Che  selection  of 
a  risk-adjusted  cost  estimate,  consideration  of  inflation  has  been  excluded 
to  this  point  for  clarity  of  presentation.  All  treatment  of  coets  and  coat 
uncertainties  has  been  in  base-year  dollars.  Cost  estimates  for  budgetary 
requirements  generelly  need  to  reflect  the  impacts  of  inflation  and  be 
expressed  in  then-year  dollars.  This  can  be  accomplished  es  described  below. 

The  RDT&E  and  production  costs  along  with  their  uncertainties  (i.e., 
standard  deviations)  can  be  spread  by  the  use  of  historieelly  derived  spread 
factors,  programmatic  schedules,  or  by  other  means,  over  the  years  in  which 
they  are  expected  to  be  incurred.  To  do  this,  the  prorated,  or  allocated 
cost  estimate  as  wall  as  its  uncertainty  estimate  for  a  given  year  can  be 
generated  by  multiplying  the  total  RDT&E  or  production  costa  and  its  cost 
uncertainty  o  by  the  appropriate  spread  factor.  The  prorated  coets 
together  with  the  cost  uncertainties  thus  generated  for  each  future  year  can 
than  be  multiplied  by  the  expected  inflation  factor  (a.g.,  Office  of  the 
Secretary  of  Defense  (08D)  inflation  factor)  for  that  year.  This  procedure 
corresponds  to  that  described  under  the  constant  multiplier  approach  in 
Section  III  E  3.  Thus  spread  (allocated)  and  inflated  cost  estimates  as  well 
as  their  uncertainties  can  be  generated  by  direct  multiplication  by  the 
spread  factors  and  inflation  factors. 

B.  ALTERNATIVE  AGGREGATIONS  AND  COST-PROBABILITY  PLOTS 


Once  the  costs  and  their  uncertainty  estimates  have  been  spread  across 
the  corresponding  years  and  inflated,  the  final  steps  are  (1)  to  aggregate 
the  costs  and  their  uncertainties  (by  adding  the  standard  deviations). 
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(2)  generate  cost-probability  plots,  and  (3)  salaet  R-ACE  values.  The  direc¬ 
tion  of  aggregation  and  the  resulting  plots  depend  on  the  cost  estimates  that 
are  being  sought.  For  example,  if  annual  funding  requirements  are  desired, 
the  aggregation  will  be  by  year;  if,  on  the  other  hand,  funding  by  program 
phase  is  desired,  aggregation  will  be  by  RDT&E  phase,  production  phase,  etc.; 
if  total  program  cost  is  desired,  all  costs  as  wall  as  their  associated  un¬ 
certainties  need  to  be  aggregated.  After  the  sought  costs,  for  example 
annual  funding  requirements  and  their  uncertainties,  are  totaled  by  year, 
than  cost-probability  curves  can  be  generated  by  year  and  provided  to  the 
contracting  agency  for  R-ACE  selection  of  funding  requests. 


VII.  FUTURE  EFFORT 


In  examining  areas  of  deficiencies  in  cost  analysis,  two  appear  to  be 
outstanding  and  so  constitute  prime  candidates  for  future  effort.  The  first 
is  the  replacement  of  the  method  of  Least  squares  by  the  method  of  least  dis¬ 
tance  for  generating  cost  estimating  relationships.  When  relationships 
developed  by  the  method  of  least  squares  are  used  to  predict  costs  in  the 
upper  ranges  of  the  independent  variable,  the  results  are  significant  under¬ 
estimates.  The  method  of  least  distance,  which  will  be  described  and  illus¬ 
trated  in  the  subsequent  effort,  does  not  possess  this  deficiency. 

The  second  area  of  deficiency  is  in  the  estimation  and  quantification  of 
key  aspects  of  schedule  risk.  Results  of  progress  on  these  two  efforts  will 
be  reported  at  the  meetings  of  the  Space  Systems  Cost  Analysis  Group  (SSCAG). 
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